1. The Field of the Invention
The present invention relates to systems and methods for gathering, organizing and executing test cases irrespective of the language or format employed. More specifically, the present invention relates to systems and methods that organize a hierarchy comprising test cases, test suites and test modules, traverse the hierarchy to run selected test cases on developed software packages to identify erroneous logic, and execute test cases on multiple threads in various scenarios.
2. The Prior State of the Art
Once a computer program has been developed, it is customary to identify and resolve any erroneous logic that was introduced during the stage of writing the program. The erroneous logic is referred to as a “bug” and includes invalid data or instructions that cause a division by zero or misdirect the computer to the wrong place in the program. Bugs cause a program to provide invalid output that may or may not result in the program crashing. Therefore, extensive testing of a newly developed computer program is required to identify any erroneous logic in the program.
One manner of testing a newly developed computer program is by applying a series of related steps, referred to as a “test case,” designed to test an aspect or feature of the developed computer program. This manner of testing is referred to as automated or programmatic testing. The test cases are repeatedly executed on the computer program by a set of instructions known as a “harness.”
Each time a computer program is tested programmatically, a new or modified harness is written to execute developed test cases on the computer program. Within teams of program testers it is common to use many different harnesses, each having been developed and maintained to service a narrow set of test cases. The developed harnesses are hard-coded to the specific language and format of the developed programs and/or test cases. Furthermore, the harnesses are frequently written in line with the test cases, causing writers of test cases to learn the challenging task of separating the code of the test cases from the code of the harness. Further still, harnesses are often inextricably tied to the user interface they expose so that a user of a harness is only provided one way to control the harness and to view the results.
As such, computer programmers currently spend large amounts of time writing or modifying a harness after a new computer program has been developed so that the newly developed program can be tested. The large amounts of time required to develop each harness have resulted in less time testing the developed program. Moreover, since each harness is customized for a narrow set of test cases, a great proliferation of customized test harnesses hard-coded to the specific language and format of the developed program and/or test case currently exist in the area of software development and testing. As a result, computer program testers have found the large number of test harnesses to be very difficult to manage.